System and method for sharing and controlling multiple audio and video streams

ABSTRACT

A user of a first device may select at least one multimedia presentation. Based on this selection, the multimedia presentation is accessed from a storage source and presented to the user. Simultaneously, information is transmitted to at least a second device that causes the selected multimedia presentation to be presented by the at least second device without the at least second device receiving input from a second user. Continuous media streams from the first user may be captured and transmitted to the second device(s). The presenting of the multimedia presentation on the first device are coordinated with the presenting of the multimedia presentation on the second device(s). The presenting of the captured media streams on the second device(s) are coordinated with the capturing of those streams and the presenting of the multimedia presentation on the second device(s). Users may control the multimedia presentations and the captured media streams.

DESCRIPTION OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention generally relates to audio and videocommunication systems, and more specifically to audio and video systemsthat enable users with independent terminal devices to share audiovisualcontent in the context of a communication session, shared softwareapplication, or common experience.

[0003] 2. Description of the Related Art

[0004] There are many audio and video communication technologies inexistence today. However, these technologies have severe limitations intheir ability integrate audiovisual content into synchronouscommunication among individuals engaged in loosely coordinated activityregardless of physical proximity. Current technologies force users tochoose between either hearing and viewing multimedia presentations orconversing with companions via audio and video. Present technologies donot provide users with the ability to integrate multimedia presentationswith their conversations to dynamically create a shared experience. Forinstance, current technologies are unable to support a group of peoplewho would like to view information about paintings in a museum but alsowant to share the experience with each other and contribute input to thegroup experience.

[0005] The media space is a technology that supports shared audio and,in some cases, audio and video communication. Examples of audio-onlymedia spaces are Somewire and Thunderwire (Singer, Hindus, Stifelman,White, “Tangible Progress: Less is more in Somewire Audio Spaces”,SIGCHI 1999, pp. 15-20, ACM). These systems do not support video and donot integrate non-microphone audio elements such as prerecorded musicand allow these to be controlled by participants. Media spaces thatsupport both audio and video also do not integrate prerecorded orsignificant generated audiovisual content and provide very littlecontrol to override shared content with personal selections, forexample. Moreover, current media space systems lack distributed control:the ability of a particular user to automatically contribute to theaudio and video experience of another user(s) without requiring anyactions from the other user(s).

[0006] There are a variety of other audio and video communicationsystems available that are also deficient in providing a dynamic,interactive, and content-enriched mechanism for individuals tocommunicate with. Multimedia Messaging Service (MMS) is the evolution ofshort message service (SMS), which is a text-based channel available onmobile phones compatible with the Groupe Speciale Mobile (GSM) wirelessstandard. MMS appears to be a multi-corporation European StandardsTelephony Institute (ETSI) initiative to increase the media that can besent among mobile devices. This system appears to serve as adistribution mechanism rather than as a system for facilitatingreal-time and dynamic interaction among individuals. MMS does not appearto support services that allow individuals to have continuousaudio/video channels available.

[0007] Audio and video mixers and multi-track recording systems allowvarious elements of audio and video to be dynamically combined; however,these systems are not symmetric, support only a broadcast form ofcommunication, and lack distributed control. Wearable computer systemssuch as NETMAN (Kortuem, Bauer, Segall, “NETMAN: The design of acollaborative wearable computer system”, Mobile Networks andApplications 4, pp. 49-58, ACM, 1999) provide wireless communication.However, these systems are limited to live participant voices and do notsupport distributed control. The Nomadic Radio system (Sawhney andSchmandt, “Nomadic Radio: Speech and Audio Interaction for ContextualMessaging in Nomadic Environments”, TOCHI, vol 7, no. 3, ACM, Sept.2000) dynamically mixes a variety of audio elements in sophisticatedways providing direct control for the user. However, this system servesas an interface for a single user to access various message streams andthus does not support synchronous audio and video communication betweenmultiple parties.

[0008] Voice Loops (Patterson, Watts-Perotti and Woods, “Voice Loops asCoordination Aids in Space Shuttle Mission Control”, Computer SupportedCooperative Work 8, pp. 353-371, Kluwer Academic Publishers,Netherlands, 1999) and similar intercom/radio type systems providesupport for multiple channels and allow for an audio conversation.However, they do not integrate audiovisual content into the conversationor support a shared application.

[0009] The Quiet Calls system (Bly, Sokoler, Nelson, “Quiet Calls:Talking Silently on Mobile Phones”, SIGCHI 2001, pp. 174-187, ACM, 2001)involves using wireless handheld terminals (e.g. cell phones). A user isable to interact with Quiet Calls through a user interface on theterminal in order to trigger pre-recorded audio clips to be played for areceiving user to hear. In this case, however, the system is designed toplay recordings of the device owner's own voice in order to managecommunication with a caller in a situation that inhibits the owner fromspeaking (e.g. in a meeting). Further, there is no integration of audiointo a synchronous conversation, there is no shared application, and thecaller does not have any control other than to hang up. In particular,the caller is not able to make selections for the owner to hear. Theaudio recordings are also not mutually informative as the owner is usingthem to send messages to the caller, not to gain any information forhimself/herself.

[0010] Multiplayer, interactive computer games are a related technologythat allow users to interact with a distributed shared application (thegame itself). Each user has his/her own terminal (a PC) and uses theuser interface of their respective device to interact with the game. Allplayers contribute by their inputs to the state and output of the game.These games typically use audio extensively to provide sound effectsthat convey significant information by indicating, for example, theproximity of another player. Some games, and companion programs likeTeamSound, have added inter-player communication features like real-timevoice conferencing, the ability to trigger playing of audio recordingsfor all players in a group, and even the ability to send text messagesthat are turned into audio by voice synthesis. However, the games aredesigned for terminals with large screens and sophisticated 3D graphicsproviding an immersive experience in a virtual environment and thus thecommunication and sharing features are not designed for portablewireless devices. Moreover, the games do not incorporate video among themultimedia content that can be shared. Although symmetric, the userselected audio recordings to play is done as in Quiet Calls simply as arapid form of message communication, not as a way to gain informationthat can be shared with others and those user selected audiopresentations are never mutually informative: where all parties involvedlearn or experience something they previously were unaware of. The gamesystems also do not offer control features that, for instance, allow oneplayer to hear a presentation they select overriding just for themselveswhat anyone else has selected.

[0011] The Etherphone system, another related work created by XeroxPARC, is described in “Etherphone: Collected Papers 1987-1988”, PARCTechnical Report CSL-89-2, May 1989. This is a system for enhancingtelephone service using computer networks and servers and computerworkstations for richer user interfaces. An Etherphone terminalincorporates a conventional telephone set, along with speaker,microphone, and computer workstation (shared with other functions).Etherphone contemplates a wide variety of features including the abilityto add voice annotations to documents or otherwise use audio in computerapplications, controlling call handling with the ability to select aperson to call from a list on the screen, automatic forwarding, customring tones, and the ability to carry on a voice conversation whileinteracting with shared collaborative applications. One of the features,Background Calls, allows parties to share a long term voicecommunication session which could be superceded by other short termcalls. Etherphone publications also speak of access to television andradio broadcasts and shared recorded audio files through the system.However, Etherphone features are linked to an office setting withcomputer workstations and wired telephones and do not address the mobilewireless context. The Etherphone system also does not include sharedapplications providing mutually informative audio or video. Moreover,Etherphone does not provide a mixture of sharing and independentcontrol.

[0012] There are various collaborative work tools like those availableto use alongside Etherphone, and remote teleconference tools likeMicrosoft's NetMeeting that support sharing regular applications on acomputer. However, these tools do not incorporate shared applicationsusing mutually informative audio, sharing and independent control, norportable wireless service.

[0013] Another set of related systems are instant messaging and chatsystems. However, these systems do not integrate audiovisual contentinto Ii conversations, nor offer the control features which allowsharing and independent control.

[0014] There are games for mobile phones, in which users have mobilewireless terminals and each provide inputs that result in the playing ofgame sounds on other devices. The game forms a shared applicationbetween the players; however, these games do not provide synchronousvoice or video communication between the players through the device anddo not include a mutually informative shared application.

[0015] Current audio and video technologies do not provide users withthe ability to dynamically integrate informative multimediapresentations with conversations to create a shared experience. Further,current systems do not allow users to automatically experience whatother users are viewing or hearing, regardless of physical proximity andwithout requiring user input, while also allowing for individualpreferences and control.

SUMMARY OF THE INVENTION

[0016] Systems, methods, apparatuses, and computer readable mediumsconsistent with the present invention allow a user of a first device toselect at least one multimedia presentation by inputting selection data.Based on this selection, the multimedia presentation is accessed from astorage source and presented to the first user. Simultaneously,information is transmitted to at least a second wireless portable devicethat causes the selected multimedia presentation to be presented by theat least second device without the at least second device receivinginput from a second user. Continuous media streams from the first usermay be captured and transmitted to the at least second device, whereinthese media streams are presented on the at least second device withoutthe at least second device receiving input from the second user. Thepresenting of the multimedia presentation on the first device arecoordinated with the presenting of the multimedia presentation on the axat least second device; and the presenting of the captured media streamson the at least second device are coordinated with the capturing ofthose streams by the first device and the presenting of the multimediapresentation on the at least second device. Further, users may controlthe content and transmission of the multimedia presentation and thecaptured media streams.

[0017] In one embodiment, a method for presenting audiovisual datastreams on a first device includes allowing a first user of the firstdevice to select at least one audiovisual data stream stored in a memoryresiding in the first device; accessing a plurality of other audiovisualdata streams selected by at least a second user of at least a seconddevice from the memory of the first device in response to access datareceived by the first device from said second device; simultaneouslypresenting on the first device the audiovisual data streams selected bythe first user and the audiovisual data streams selected by the at leastsecond user; and distinguishing on the first device each of theaudiovisual data streams selected by the first user and each of theplurality of audiovisual data streams selected by the at least seconduser, wherein accessing the plurality of audiovisual data streamsselected by the at least second user is performed automatically by thefirst device and without input from the first user to the first device.

[0018] Additional objects and advantages of the invention will be setforth in part in the description which follows, and in part will beobvious from the description, or may be learned by practice of theinvention. The objects and advantages of the invention will be realizedand attained by means of the elements and combinations particularlypointed out in the appended claims.

[0019] It is to be understood that both the foregoing generaldescription and the following detailed description are exemplary andexplanatory only and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in andconstitute a part of this specification, illustrate one embodiment ofthe invention and together with the description, serve to explain theprinciples of the invention.

[0021]FIG. 1 is a network diagram of a network in which systems andmethods consistent with the present invention may be implemented;

[0022]FIG. 2 is a detailed block diagram representative of a devicedepicted in the network of FIG. 1;

[0023]FIG. 3 is a flow chart depicting the initialization ofpresentations on a device consistent with principles of the presentinvention; and

[0024]FIG. 4 is a flow chart depicting device operation subsequent tothe initialization of play consistent with principles of the presentinvention.

DETAILED DESCRIPTION

[0025] In the following detailed description reference will be made tothe accompanying drawings in which is shown by way of illustration aspecific embodiment in which the invention may be practiced. Thisembodiment is described in sufficient detail to enable those skilled inthe art to practice the invention and it is to be understood that otherembodiments may be utilized and that structural changes may be madewithout departing from the scope of present invention. The followingdetailed description is, therefore, not to be taken in a limited sense.

[0026] Systems, methods, and devices consistent with the presentinvention enhance conventional synchronous communication between peopleby integrating audiovisual content into natural conversation throughcoordination of such content with the activity of the individuals,especially related to a shared application or task. In one embodiment ofthe present invention, a system may comprise two or more terminalscoupled to a network, where each device may comprise audio and videoinputs and outputs; a memory including an interactive program, a controlfacility, and audiovisual content; a user interface for allowing theusers to access the control facility and interactive program; and aprocessor for executing the programs. The terminals allow users tocoordinate, communicate, and control audiovisual content stored inmemory and audio and video data streams received from users.Coordination will refer, hereinafter, to audiovisual content beingautomatically shared and presented among participants while allowing theprecise timing and form of the presentation to be adjusted in accordancewith individual preferences and actions. For example, when twoparticipants are using a shared application that provides informationabout an art exhibit, coordination may include each person automaticallyand simultaneously seeing and hearing the selected audiovisual contentabout the exhibit, while each participant retains individual priority:if both participants choose different selections at the same time(users' action), each individual will hear only their own selection inresponse. The present invention supports a shared experience anddistributed control, since no action is required for one party to hearand see what other parties are experiencing. Moreover, individualindependence is preserved through independent user control over thepresentation of the audiovisual content. The combination of integratedsharing with independent user control supports extensive audiovisualconversation among participants, and facilitates the inclusion ofaudiovisual content into informal person-to-person communication.

[0027] One implementation of the present invention may involve a groupof people visiting an art museum that provides audio and/or videocommentary on exhibits through a shared application accessible by orresiding on wireless devices. In this scenario, the group members havethe ability to communicate with each other using audio and/or videowhile simultaneously listening to the audiovisual descriptions andviewing the exhibits. Each participant may hear, through theirrespective device, descriptions played by other members of the groupwithout having to input any commands to their respective devices. Thus,each member of the group may be aware of the focus of other member'sattention and have the ability discuss descriptions that have beenexperienced. The present invention provides the above-mentionedfunctionality regardless of the physical location of the group members.

[0028] Another implementation of the present invention may include amultimedia history encyclopedia integrated with a shared voiceconference mechanism to a group of students each using a home computer.Each student may be able to select various audiovisual entries in theencyclopedia, such as recordings of voice actors reading historicspeeches or video segments of battlefield reenactments. Moreover, eachstudent's selection may be automatically played for all the others. Inthis embodiment, each student is provided with individual controlfeatures as well. These controls may include the ability for each personto turn off content from selected others, allowing the group to divideinto a number of subgroups.

[0029] It should be understood that the foregoing implementations areexemplary and should not be construed in a limiting sense. One skilledin the art will appreciate that the present invention will have avariety of implementations not limited to those previously described.

[0030] Referring now to the drawings, in which like numerals representlike elements throughout the figures, the present invention will bedescribed.

[0031]FIG. 1 is an overview of system 10 that enables users todynamically control and exchange multiple audio and/or video channelssimultaneously in accordance the principles of the present invention. Inone embodiment, the system 10 of FIG. 1 comprises at least a first userterminal 103 and a second terminal 111 coupled together via network 105.While FIG. 1 shows network 105 as comprising two user terminals, oneskilled in the art will understand that network 105 may consist of anynumber of geographically dispersed user terminals. One skilled in theart will also understand that network 105 may be the Internet or othercomputer network such an intranet, or other similar network that allowsmultiple user terminals to transmit and receive multiple data streams.User terminals 103 and 111 are operatively connected to network 105 bycommunication devices and software known in the art, such as an InternetService Provider (ISP) or an Internet gateway.

[0032] In one embodiment, terminals 103 and 111 are portable hand-heldwireless devices that provide computing and information storage andretrieval capabilities to users. As shown in FIG. 1, terminals 103 and111 may be connected to one or more audio sources 101 and 113, and videosources 109 and 125. These sources provide audio and video data streams,respectively to terminal 103 and 111 via connections 104, 108, 119, and127. Audio sources 101 and 113 may include general microphones, personalmicrophones including conventional and unconventional audio transfer(e.g. bone conduction), and line inputs from other electronic audioequipment. Video sources 109 and 125 may include cameras, videorecorders or any other image capture devices, or line inputs from otherelectronic video equipment. The audio and video sources may also be datastreams originating outside the system, such as pre-recorded televisionand live radio broadcast signals.

[0033] Terminals 103 and 111 are also connected to audio output devices107 and 115, and video output devices 110 and 120. Audio output devicesmay include, but are not limited to audio speakers, headphones, andunconventional audio transfer. Video output devices 110 and 120 mayinclude televisions, video recording devices, or any other electronicvideo equipment. For the sake of brevity, each terminal in FIG. 1 isshown to be connected to a single audio and video source and output.However, the terminals may be connected to several different sources andoutputs without departing from the scope of the present invention.Moreover, the terminals may be connected to an audio source(s) only or avideo source(s) only. Further, the audio and video sources and outputdevices may be external (shown in FIG. 1) or internal to the terminals.For instance, an LCD display or an internal speaker on the device.

[0034] Referring now to FIG. 2, it is shown that each user terminal (103and 111) is comprised of a network interface 205, user interface 210,data input port 222, data output port 224, processor 240, and memory245. Data input port 222 may further comprise an audio input port 220and a video input port 230. Data output port 224 may further comprise anaudio output port 225 and video output port 235. The terminal includesconnectors 104, 106, 108, and 112 for connecting audio and video sources101, 107, 109, and 110 to audio input 220, audio output 225, video input230, and video output 235, respectively. Memory 245 may include staticRAM and may further comprise an audio disk 247 and a video disk 248.Memory 245 further comprises audiovisual content 255, control facility260 and a set of instructions collectively referred to as interactiveprogram 270.

[0035] Interactive program 270 is run on processor 240 and controls theaudiovisual content 255 residing in memory 245. Audiovisual content 255may include, but is not limited to multimedia presentations created toinform users about their surroundings (e.g. art exhibits). Interactiveprogram 270 is responsible for the coordination of the audiovisualcontent 255. Interactive program 270 may also synchronize audio andvideo data streams received from audio source 101 and video source 109with the audiovisual content 255 stored in memory. Interactive program270 may be implemented in software that resides in memory 245 (as shown)or embedded on processor 240.

[0036] Control facility 260 interacts with interactive program 270 andprovides users with the ability to modify and mix the audio and videodata streams received from audio source 101 and video source 109 and theaudiovisual content in order to create an output data stream to send toanother terminal via network interface 205. Mixing may include, but isnot limited to, spatial variation, loudness variation, and effectvariation. Spatial variation may involve arranging audio streams in sucha way as to create a simulated sound field for the user. Effectvariation may involve adding reverb, delay, echo, gating and othereffects to audio and altering the hue of a visual display. In oneembodiment, these mixing techniques are performed by the sendingterminal. However, in another embodiment, these mixing techniques may beperformed by the receiving terminal. Control facility 260 provides userswith the ability to establish a communication session: to select whichterminal to share audiovisual content 255 and audio and video datastreams received from audio source 101 and video source 109 with. Thismay also involve subdividing further into sub-sessions. Control facility260 also provides dynamic control over the transmission of the outputdata stream to the selected terminals. This control may include, but isnot limited to: adding additional users to a session, removing usersfrom a session, controlling the volume levels, and selecting specificportions of the audiovisual content 255 to transmit. For instance, auser may wish to send one part of a particular image scene (e.g., asingle flower from an image of a bouquet of flowers), which can bespecified using the controls. Control facility 260 also allows a 11 userto decline and consent to output data streams produced by other usersand received via network interface 205. Control may include, but is notlimited to, outright accepting, outright refusing, receiving portions,and specifying specific users to refuse or accept channels from. Userscan establish access policies prior to receiving the channels or mayrefuse and accept channels subsequent to their arrival. Additionally,control facility 260 allows a user to customize the presentation (onaudio output 107 and video output 110) of the audiovisual content 255stored in memory 245 and output data streams produced by other terminalsand received via network interface 205. Control facility 260 may beexecuted on processor 240 and may be implemented in software thatresides in memory 245 (as shown) or embedded on processor 240.

[0037] The foregoing descriptions of control facility 260 andinteractive program 270 are exemplary only. The specific functionsprovided by each of these functional blocks may be interchangeable.Further, in another embodiment, these functional blocks may be combinedinto a single block. Moreover, interactive program 270 and audiovisualcontent 255 may reside on a server remotely located with respect to theuser terminal. Additionally, the functions of control facility 260 andinteractive program 270 may be automatically provided to the terminalwithout user input.

[0038] Users may access control facility 260 and interactive program 270and issue specific control commands, which control facility 260 andinteractive program 270 provide, via user interface 210. User interface210 may be an input port connected by a wired, optical, or a wirelessconnection for electromagnetic transmissions, or alternatively may betransferable storage media, such as floppy disks, magnetic tapes,compact disks, or other storage media including the input data from theuser. User interface 210 may also include at least one button actuatedby the user to input commands to select from a plurality of operatingmodes to operate processor 240. In alternative embodiments, userinterface 210 may include a keyboard, a mouse, a touch screen, and/or adata reading device such as a disk drive for receiving the input datafrom input data files stored in storage media such as a floppy disk orother storage tape. User interface 210 may alternatively includeconnections to other computer systems to receive the input commands anddata therefrom. Processor 240 is operatively configured to receive inputcommands and data from a user associated with the user terminal thoughuser interface 210.

[0039] Terminal 103, as shown in FIG. 2, is connected to network 105 vianetwork interface 205, which is operatively connected via a wired andwireless communications link. Network interface 205 may be a networkinterface card, unit, or any other type of dedicated network connection.In operation, network interface 205 may be used to send data to andreceive data from other terminals.

[0040] Referring now to FIG. 3, a detailed flowchart depicting theinitialization of play on a terminal will be described. It should beunderstood that in operation the particular order of events may varyfrom this description without departing from the scope of the presentinvention.

[0041] In operation, a communication session is established betweenterminal 103 and any number of additional like terminals via controlfacility 260. The session may be established, for example, by enteringnetwork ID's into each terminal for identifying members. Once a sessionis established, the audio and video streams received from audio source101 and video source 109 are transmitted via network interface 205 tothe other terminals in the session. Each terminal in the sessionreceives these data streams, controls them by way of their respectivecontrol facilities, and presents them to the respective users. Thissession establishment and person-to-person audio and video datacommunication takes place in the idle state, indicated by step 300. Thisstep may also involve applying various compression and feedbackreduction techniques, by way of control facility 260. In one embodiment,informative multimedia presentations (e.g. audiovisual content 255) arestored locally in the respective memories of each terminal andaccessible by each of the respective users. A multimedia presentationstored in memory 245 selected by a user of terminal 103 for playing onterminal 103 will hereinafter be referred to as a foregroundpresentation. Whereas, a presentation selected and triggered by anothersession member for play on terminal 103 will hereinafter be referred toas a background presentation. A user is able to control both thebackground and foreground presentations by way of control facility 260via user interface 210. Control may include, but is not limited to:volume level control and turning specific background presentations onand off. Further, interactive program 270 and control facility 270 mayadd audio and video effects to the presentations. As indicated in step310, a user may select a locally stored foreground presentation forplaying. Selecting may include, but is not limited to tapping an objectin an image presented via user interface 210. In another embodiment, aforeground selection may occur automatically by way of a user's physicalproximity to a particular object in the environment. As indicated instep 312, the selection of a foreground presentation causes acorresponding PLAY message to be transmitted to all of the sessionmembers indicating the time at which the play started, the particularpresentation being played, and the source terminal identifier. Asindicated in step 315, the selection triggers the access of the selectedpresentation from memory 245. Step 350 indicates that the selected andaccessed foreground presentation is being presented to the user. Thepresenting (or playing) of presentations may be performed via audio andvideo output devices residing internally or externally with respect toterminal 103. In one embodiment, the playing of a presentation may alsobe performed in part or fully by a display mechanism resident in theuser interface. Conversely, when terminal 103 receives a PLAY messagefrom another sending terminal, as indicated in step 320, terminal 103records it in a portion of memory 245 previously designated for thatsending device, as specified in step 325. As indicated in step 327,terminal 103 then determines, based on the user's control settings andterminal state, whether to play the presentation. If the background hasnot been disabled by the user, the background presentation selected bythe sending user will be accessed by terminal 103 (step 329) andimmediately played on terminal 103 (step 350), thereby allowing theusers to experience the presentation simultaneously. In this embodiment,all presentations are locally stored and, therefore, a backgroundpresentation is accessed by terminal 103 via interactive program 270using pointers or file names received from the sending device(s) throughnetwork interface 205. If at step 327 it is determined that thebackground presentation cannot be played on terminal 103 immediately,terminal 103 returns to the idle state (step 300). However, if thebackground is enabled by the user at a later time (step 340) and thebackground presentation was previously recorded in memory (step 345) andavailable (step 347), interactive program 270 will access the backgroundpresentation from memory 245 and seek forward the amount of time elapsedsince the play of the background presentation commenced and will beginplay of the background presentation on terminal 103 from that point(steps 349 and 350), thereby allowing the users to experience thepresentation simultaneously even though the user of terminal 103experiences only a portion of the presentation. Step 349 may alsoinclude allowing the user to input control settings for controlling thebackground presentation. If more than one background presentation waspreviously sent to terminal 103, then interactive program, in oneembodiment, chooses one presentation to access and play, as indicated instep 348. This choosing may be controlled by preset algorithms residentin interactive program 270 and may include cueing the presentations in aparticular order.

[0042] For the sake of brevity, the foregoing description of FIG. 3assumes that terminal 103 is plays either a foreground presentation or abackground presentation at step 350. However, in another embodiment,multiple foreground and/or background presentations are playedsimultaneously. For example, terminal 103 could receive multiple PLAYmessages from more than one sending terminal corresponding to severaldifferent background presentations, or a user may select more than oneforeground presentation for simultaneous play. In such cases,interactive program may distinguish the various presentations onterminal 103 by way of effect, spatial, and loudness variation. Foraudio, distinguishing may involve adding effects including reverb,delay, gating, and echo. Distinguishing may also involve spatialpositioning to create a multidimensional sound field, pitch shifting,bandpass filtering, treble and bass boosts and cuts, and addingdistinctive sounds in the background. For video, distinguishing mayinvolve color shifting: black and white, sepia tone, varyingsaturations, size variation, filtering, cropping, and adding andsuperimposing distinctive images. In one embodiment, the effectsprocessing is done in advance and each device contains multiple versionsof a presentation in their respective memories. In another embodiment,the effects processing is done by the respective processors as theaudiovisual content is being played.

[0043] Referring now to FIG. 4, a detailed flowchart depicting terminaloperation subsequent to the initialization of a foreground or backgroundpresentation will be described. It should be understood that inoperation the particular order of events may vary from this descriptionwithout departing from the scope of the present invention.

[0044] As previously indicated, step 350 may involve playing either aforeground or background presentation or, in another embodiment, somecombination thereof. If the user of terminal 103 selects a foregroundpresentation (step 410) after terminal 103 has left the idle state (i.e.a previously chosen or received presentation is being played),interactive program 270 may perform a priority conflict check (step412). In one embodiment, foreground presentations have priority overbackground presentations. Therefore, if it is determined at step 412that a background presentation is currently being played, theninteractive program 270 will halt play of that presentation, access theforeground presentation from memory 245, and play the accessedforeground presentation. If there is no priority conflict, theforeground presentation will be accessed and played immediately. Ifterminal 103 receives a STOP message (step 420), the correspondingrecord will be removed from memory, as indicated by step 422. If thebackground presentation corresponding to the STOP message is playing(step 424), interactive program will halt its play, as indicated in step426. Interactive program will then search for any other backgroundpresentation available for play, whether cued or otherwise (step 428).If another background presentation is found, interactive program willaccess and seek forward the amount of time elapsed since the play ofthat background presentation commenced and will begin play of thebackground presentation on terminal 103 from that point (step 461).However, if no other background presentations are to be played, terminal103 will return to idle (step 300). Step 430 indicates another PLAYmessage being received by terminal 103. As explained in connection withFIG. 3, the message will be recorded in memory 245, as indicated in step432. However, at this point a priority check is performed (step 434) toensure that a foreground presentation is not currently playing. If thereis no conflict, the background presentation corresponding to the PLAYmessage will be accessed and played. However, if there is a priorityconflict, the foreground will continue to be played and the backgroundwill be handled by interactive software 270. In one embodiment, thebackground presentation may be placed in a cue. Step 440 indicates thata user of terminal 103 has disabled the background on the terminal. Thiswill cause interactive program 270 to check if there is a backgroundpresentation playing, as specified in step 444. If there is nobackground presentation playing and there is a foreground presentationbeing played, the terminal will return to step 350 and continue playingthe foreground presentation. However, if a background presentation isbeing played, it will be halted (step 446) and terminal 103 will returnto the idle state (step 300). Step 450 indicates that either abackground or foreground presentation has ended. In response, terminal103 will stop playing (step 452). If there was a foreground presentationplaying, terminal 103 will transmit a corresponding STOP message to allother session members, as indicated in step 456. If there was abackground presentation playing, interactive program will search forother background presentations to be played as indicated in step 458. Ifanother background presentation is found, interactive program willaccess and seek forward the amount of time elapsed since the play of thebackground presentation commenced and will begin play of the backgroundpresentation on terminal 103 from that point, as indicated in step 461.

[0045] For the sake of brevity, the foregoing description of FIG. 4assumes that only one background presentation is played at a time andthat foreground presentations take priority over backgroundpresentations. However, those skilled in the art will appreciate thatterminal 103 can be configured to mix and display a plurality ofbackground and foreground presentations simultaneously, and to audiblyor visually differentiate the various presentations for the user.Moreover, in another embodiment, it may prove advantageous to givebackground presentations priority over foreground presentations. Itshould also be understood that person-to-person communication is notconfined to the idle state. The present invention integratesperson-to-person conversations with the playing of multimediapresentations. Thus, none of the above-mentioned steps preclude usersfrom simultaneously conversing with each other via audio and videostreams.

[0046] In operation, control facility 260 and interactive program 270may provide users with different modes of synchronization including:peek and slave. Peeking is a temporally short length of time when a useris able see and hear what another user is viewing or listening to.Slaving is a temporally long length of time when a user is able toconnect to another device and present what that user's terminal ispresenting. The terminal remains slaved until the user issues a cancelcommand through user interface 210.

[0047] In operation, each terminal may be adapted to engage in push andpull modes of sharing. Pull sharing involves the sender asking foraudiovisual content, while push focuses on sending and displayingaudiovisual content to another device. This push and pull model allowsusers to share information with each other and also attract anotherusers' attention.

[0048] The controls, inputs and other constraints that determine theinteractions between a sending and receiving terminal may be referred toas the profile of a particular communication session. In accordance withone aspect of the present invention, each user participates in onecommunication session per terminal. In another embodiment, a user may beable to participate in multiple sessions using a single terminal. Asession profile may also be dynamic: it may be composed of manysub-profiles and the composition may change over time as a result of theactions of the participants and the environment.

[0049] It should be understood that programs, methods etc. describedherein are not inherently related to any particular computer or otherapparatus, and may be implemented by a suitable combination of hardware,software and/or firmware. Further, various types of general purposemachines may be used with programs constructed in accordance with theteachings described herein. Similarly, it may be advantageous toconstruct specialized apparatuses to perform the method steps describedherein.

[0050] It will be apparent to those skilled in the art that variousmodifications and variations can be made in the systems, methods anddevices of the present invention and in the construction of thisinvention without departing from the scope of or spirit of theinvention. For example, the control facility and interactive program maybe integrated into one collective software package. Further, thefunctions provided by the control facility and interactive program maybe interchangeable and may be provided without user inputs. Moreover,the interactive program may reside at a remote location rather thanbeing distributed among each user terminal. Additionally, data may betransmitted and received between terminals directly rather than via anetwork.

[0051] The present invention has been described in relation to aparticular embodiment which is intended in all respects to beillustrative rather than restrictive. Those skilled in the art willappreciate that many different combinations of hardware, software,and/or firmware will be suitable for practicing the present invention.

[0052] Moreover, other embodiments of the invention will be apparent tothose skilled in the art from consideration of the specification andpractice of the invention disclosed herein. It is intended that thespecification and examples be considered as exemplary only, with a truescope and spirit of the invention being indicated by the followingclaims.

What is claimed is:
 1. A method in a first wireless portable devicecomprising: allowing a first user of the first device to select at leastone multimedia presentation by inputting selection data; accessing theselected multimedia presentation from a storage source and presentingthe selected multimedia presentation to the first user in response tothe selection; transmitting information to at least a second wirelessportable device that causes the selected multimedia presentation to bepresented by the at least second device without the at least seconddevice receiving input from a second user or the first device receivingadditional data from the first user; capturing continuous media streamsfrom the first user and transmitting the media streams to the at leastsecond device, wherein the media streams are presented on the at leastsecond device without the at least second device receiving input fromthe second user or the first device receiving additional data from thefirst user; coordinating the presenting of the multimedia presentationon the first device with the presenting of the multimedia presentationon the at least second device; coordinating the presenting of thecaptured media streams on the at least second device with the capturingof said streams by the first device and the presenting of the multimediapresentation on the at least second device; and dynamically controllingthe content and transmission of the multimedia presentation and thecaptured media streams.
 2. The method of claim 1, wherein transmittinginformation to at least a second wireless portable device includestransmitting the multimedia presentation selected by the first user. 3.The method of claim 1, wherein transmitting information to at least asecond wireless portable device includes transmitting access data thatallows the at least second device to access the multimedia presentationselected by the first user from the storage source.
 4. The method ofclaim 1, wherein coordinating the presenting of the multimediapresentation by the first device with the presenting of the multimediapresentation on the at least second device includes presenting themultimedia presentation on the first and second devices simultaneously,presenting the multimedia presentation on the at least second device atthe earliest time after presenting the multimedia presentation on thefirst device when the at least second device is not busy, and presentingthe multimedia presentation on the at least second device at apre-calculated time.
 5. The method of claim 1, wherein coordinating thepresenting of the captured media streams on the at least second devicewith the capturing of said streams by the first device and thepresenting of the multimedia presentation on the at least second deviceincludes presenting the captured media streams on the at least seconddevice immediately, in the same time relation to the presenting of themultimedia presentation as it had to the presenting of the multimediapresentation on the first device, or a calculated combination thereof.6. The method of claim 1, wherein accessing the selected multimediapresentation from a storage source involves accessing the multimediapresentation from a memory device residing internally with respect tothe first device.
 7. The method of claim 1, wherein accessing theselected multimedia presentation from a storage source involvesaccessing the multimedia presentation from an external storage mediumcoupled to the first device.
 8. The method of claim 1, wherein accessingthe selected multimedia presentation from a storage source involvesaccessing the multimedia presentation from an external source accessiblevia a wireless communication link.
 9. The method of claim 1, whereincoordinating includes mixing the multimedia presentation and thecaptured media streams.
 10. The method of claim 1, wherein dynamicallycontrolling includes inhibiting the at least second device fromreceiving the multimedia presentation.
 11. The method of claim 1,wherein dynamically controlling includes selecting portions of themultimedia presentation to transmit to the at least second device. 12.The method of claim 1, wherein capturing continuous media streamsincludes capturing voice signals from the user.
 13. The method of claim1, wherein capturing continuous media streams includes capturing videosignals from an environment.
 14. A computer readable medium containing aset of instructions for performing a method in a system comprising aplurality of users operating a plurality of devices, the methodcomprising: communicating a mutually informative multimedia presentationto the plurality of devices in response to a trigger received from atleast one of the plurality of devices; allowing each of the plurality ofdevices to automatically present the mutually informative multimediapresentation without the users of the devices providing input to thedevices; and allowing the users of the plurality of devices to controlthe mutually informative multimedia presentation on their respectivedevices.
 15. The computer readable medium of claim 14, wherein the setof instructions is remotely located with respect to the plurality ofdevices.
 16. The computer readable medium of claim 14, wherein the setof instructions is distributed and coordinated among the plurality ofdevices.
 17. The computer readable medium of claim 14, whereincontrolling the mutually informative multimedia presentation on theplurality of devices includes coordinating the presenting of themutually informative multimedia presentation on the plurality ofdevices.
 18. The computer readable medium of claim 17, whereincoordinating involves presenting the mutually informative multimediapresentation simultaneously.
 19. The computer readable medium of claim17, wherein coordinating involves presenting the mutually informativemultimedia presentation on each of the plurality of devices at specifictimes calculated by the application.
 20. A system, comprising aplurality of user devices, coupled to a network, each of the pluralityof devices comprising a memory including a program that allows theplurality of devices to: receive at least one continuous data streamfrom a user of the device, select at least one other device from theplurality of devices and transmit the at least one data stream via thenetwork to the selected device, trigger the transmission of at least onemutually informative multimedia presentation from a shared interactiveapplication to the plurality of devices via the network; and dynamicallycontrol the content and transmission of the at least one data stream andthe at least one multimedia presentation.
 21. A first wireless portabledevice comprising: a wireless communication interface adapted totransmit data to at least a second wireless portable device and receivedata from the at least second device; means for producing sound that canbe heard by a first user of the first device; means for displaying stilland moving pictures that can be viewed by the first user; an audiocapture device for capturing audio signals from the first user; a videocapture device for capturing video signals from the first user; a memoryincluding a program and at least one multimedia presentation; aprocessor that runs the program; and a user interface for allowing thefirst user to input commands to the program, wherein the program: allowsthe first user to select the at least one multimedia presentation,accesses the selected multimedia presentation from the memory inresponse to the first user's selection, presents the selected multimediapresentation to the first user via the means for displaying still andmoving pictures and the means for producing sound, transmits informationto the at least second wireless portable device via the wirelesscommunication interface that causes the selected multimedia presentationto be presented by the at least second device without the at leastsecond device receiving commands from a second user or the first devicereceiving additional commands from the first user, transmits the audioand video signals from the audio and video capture devices to the atleast second device via the wireless communication interface, whereinthe captured audio and video signals are presented on the at leastsecond device without the at least second device receiving commands fromthe second user or the first device receiving additional commands fromthe first user, coordinates the presenting of the multimediapresentation on the first device with the presenting of the multimediapresentation on the at least second device, and coordinates thepresenting of the captured audio and video data on the at least seconddevice with the capturing of said data by the first device and thepresenting of the multimedia presentation on the at least second device,and allows the first user to dynamically control the content andtransmission of the multimedia presentation and the captured audio andvideo signals.
 22. The first device of claim 21, wherein the informationtransmitted to the at least second wireless portable device that causesthe selected multimedia presentation to be presented includes themultimedia presentation selected by the first user.
 23. The first deviceof claim 21, wherein the information transmitted to the at least secondwireless portable device that causes the selected multimediapresentation to be presented includes access data that allows the atleast second device to access the multimedia presentation selected bythe first user from memory on the at least second device.
 24. The firstdevice of claim 21, wherein the program allows the first user todynamically inhibit the at least second device from receiving themultimedia presentation by issuing a command via the user interface. 25.The first device of claim 21, wherein the program allows the first userto dynamically select portions of the multimedia presentation totransmit to the at least second device in response to user commands. 26.The first device of claim 21, wherein the program allows the first userto dynamically control the volume levels of the multimedia presentation.27. The first device of claim 21, wherein the program coordinates thepresenting of the multimedia presentation by the first device with thepresenting of the multimedia presentation on the at least second deviceby presenting the multimedia presentation on the first and seconddevices simultaneously.
 28. The first device of claim 21, wherein theprogram coordinates the presenting of the multimedia presentation by thefirst device with the presenting of the multimedia presentation on theat least second device by presenting the multimedia presentation on theat least second device at the earliest time after presenting themultimedia presentation on the first device when the at least seconddevice is not busy.
 29. The first device of claim 21, wherein theprogram coordinates the presenting of the multimedia presentation by thefirst device with the presenting of the multimedia presentation on theat least second device by presenting the multimedia presentation on theat least second device at a time calculated by the interactive softwareapplication.
 30. The first device of claim 21, wherein the programcoordinates the presenting of the captured audio and video data on theat least second device with the capturing of said data by the firstdevice and the presenting of the multimedia presentation on the at leastsecond device by presenting the captured audio and video data on the atleast second device in the same time relation to the presenting of themultimedia presentation as it had to the presenting of the multimediapresentation on the first device.
 31. The first device of claim 21,wherein the program coordinates the presenting of the captured audio andvideo data on the at least second device with the capturing of said databy the first device and the presenting of the multimedia presentation onthe at least second device by presenting the captured audio and videodata on the at least second device at a time calculated by theinteractive software application.
 32. A first wireless portable devicecomprising: a wireless communication interface adapted to transmit datato at least a second wireless portable device and receive data from theat least second device; means for producing sound that can be heard by afirst user of the first device; means for displaying still and movingpictures that can be viewed by the first user; a memory including aprogram and at least one multimedia presentation; a processor that runsthe program; and a user interface for allowing the first user to inputcommands to the program, wherein the program: allows the first user toselect the at least one multimedia presentation, accesses the selectedmultimedia presentation from the memory in response to the first user'sselection, presents the selected multimedia presentation to the firstuser via the means for displaying still and moving pictures and themeans for producing sound, transmits information to the at least secondwireless portable device via the wireless communication interface thatcauses the selected multimedia presentation to be presented by the atleast second device without the at least second device receivingcommands from a second user or the first device receiving additionalcommands from the first user, coordinates the presenting of themultimedia presentation on the first device with the presenting of themultimedia presentation on the at least second device, and a controlfacility for allowing the first user to dynamically control the contentand transmission of the multimedia presentations.
 33. A method forpresenting audiovisual data streams on a first device, the methodcomprising: allowing a first user of the first device to select at leastone audiovisual data stream stored in a memory residing in the firstdevice; accessing a plurality of other audiovisual data streams selectedby at least a second user of at least a second device from the memory ofthe first device in response to access data received by the first devicefrom said second device; simultaneously presenting on the first devicethe audiovisual data streams selected by the first user and theaudiovisual data streams selected by the at least second user; anddistinguishing on the first device each of the audiovisual data streamsselected by the first user and each of the plurality of audiovisual datastreams selected by the at least second user, wherein accessing theplurality of audiovisual data streams selected by the at least seconduser is performed automatically by the first device and without inputfrom the first user to the first device.
 34. The method of claim 33,wherein accessing includes using pointers and file names to access themultimedia presentations selected by the at least second user.
 35. Themethod of claim 33, wherein distinguishing includes effect variation.36. The method of claim 33, wherein distinguishing includes spatialvariation.
 37. The method of claim 36, wherein spatial variationincludes arranging the audiovisual data streams in such a way as tosimulate a multi-dimensional sound field.
 38. The method of claim 35,wherein effect variation includes adding reverb, echo, delay, and gatingto the audiovisual streams.
 39. The method of claim 33, whereindistinguishing includes adding distinctive synthesized sound and videosegments to the audiovisual streams.